rustc: fix building with llvm #320432

RossComputerGuy · 2024-06-17T05:31:33Z

Description of changes

Fixes #311930

Can be tested / reproduced by building pkgsLLVM.rustc. Fix was tested under the Nixpkgs LLVM Workspace project.

Things done

Add a 👍 reaction to pull requests you find important.

tomberek · 2024-06-20T17:02:23Z

I still get this on x86_64-linux:

      >   = note: /nix/store/wwfrj9kvfi14xclc38qfwm71ah6aawdh-binutils-2.41/bin/ld: cannot find -lgcc_s: No such file or directory
       >           clang: error: linker command failed with exit code 1 (use -v to see invocation)
       >
       >
       > error: could not compile `typenum` (build script) due to 1 previous error
       > failed to run: /nix/store/qyw7qc22j2ngf9wip8sxagaxb0387gnq-cargo-1.78.0/bin/cargo build --manifest-path /build/rustc-1.78.0-src/src/bootstrap/Cargo.toml --frozen

RossComputerGuy · 2024-06-20T17:04:55Z

@tomberek Yes, this seems to still occur on aarch64-linux as well. Not entirely sure why we are still facing this issue. Looking at the documentation comments in config.example.toml in rust, it should work. Likely an upstream issue with rustc not detecting things right?

RossComputerGuy · 2024-06-22T17:44:56Z

@ofborg build pkgsLLVM.rustc

jopejoe1 · 2024-06-22T20:09:11Z

On x86_64-linux I'm getting:

       >   = note: /nix/store/wwfrj9kvfi14xclc38qfwm71ah6aawdh-binutils-2.41/bin/ld: cannot find -lstdc++: No such file or directory
       >           clang: error: linker command failed with exit code 1 (use -v to see invocation)
       >           
       >
       > error: could not compile `rustix` (build script) due to 1 previous error
       > error: linking with `/nix/store/pyhikk3r5zr1h5mg06k94j9bsl2lgyqv-x86_64-unknown-linux-gnu-clang-wrapper-17.0.6/bin/x86_64-unknown-linux-gnu-clang` failed: exit status: 1

RossComputerGuy · 2024-06-22T20:45:47Z

@jopejoe1 Thank you for testing, I marked it as a draft and will continue figuring out why rustc still doesn't compile tonight.

paparodeo · 2024-06-22T20:58:34Z

On x86_64-linux I'm getting:

       >   = note: /nix/store/wwfrj9kvfi14xclc38qfwm71ah6aawdh-binutils-2.41/bin/ld: cannot find -lstdc++: No such file or directory

perhaps we should be compiling the pkgsLLVM clang so that it translates the -lstdc++ to -lc++ by default. this is the behavior for systems where the default c++ library is -lc++. there is somewhat of a discussion about this translation as it relates to darwin: #273016 (comment)

alternatively, adding the flag -stdlib=libc++ should also work

RossComputerGuy · 2024-06-23T00:55:35Z

@paparodeo https://github.com/NixOS/nixpkgs/blob/b966a22b42e71914c65d9a2c0635e2d0f7d7a106/pkgs/development/compilers/rust/rustc.nix#L61 🤔

RossComputerGuy · 2024-07-05T21:43:47Z

Rustc builds now 🎉

(Will let CI run through and if labels do not change, will retarget for staging)

RossComputerGuy · 2024-07-05T22:53:19Z

@ofborg build pkgsLLVM.rustc

Mic92 · 2024-07-06T06:29:10Z

pkgs/development/compilers/rust/rustc.nix

@@ -66,14 +67,14 @@ in stdenv.mkDerivation (finalAttrs: {
       # when linking stage1 libstd: cc: undefined reference to `__cxa_begin_catch'
       # This doesn't apply to cross-building for FreeBSD because the host
       # uses libstdc++, but the target (used for building std) uses libc++
-       optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD) "--push-state --as-needed -lstdc++ --pop-state"
+       optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && !useLLVM) "--push-state --as-needed -lstdc++ --pop-state"
+    ++ optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && useLLVM) "--push-state --as-needed -L${llvmPackages.libcxx}/lib -lc++ -lc++abi -lLLVM-${lib.versions.major llvmPackages.llvm.version} --pop-state"


Suggested change

++ optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && useLLVM) "--push-state --as-needed -L${llvmPackages.libcxx}/lib -lc++ -lc++abi -lLLVM-${lib.versions.major llvmPackages.llvm.version} --pop-state"

++ optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && useLLVM)

"--push-state --as-needed -L${llvmPackages.libcxx}/lib -lc++ -lc++abi -lLLVM-${lib.versions.major llvmPackages.llvm.version} --pop-state"

Mic92 · 2024-07-06T06:29:30Z

pkgs/development/compilers/rust/rustc.nix

@@ -66,14 +67,14 @@ in stdenv.mkDerivation (finalAttrs: {
       # when linking stage1 libstd: cc: undefined reference to `__cxa_begin_catch'
       # This doesn't apply to cross-building for FreeBSD because the host
       # uses libstdc++, but the target (used for building std) uses libc++
-       optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD) "--push-state --as-needed -lstdc++ --pop-state"
+       optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && !useLLVM) "--push-state --as-needed -lstdc++ --pop-state"


Suggested change

optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && !useLLVM) "--push-state --as-needed -lstdc++ --pop-state"

optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && !useLLVM)

"--push-state --as-needed -lstdc++ --pop-state"

nixos-discourse · 2024-07-18T18:30:46Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/4278

tomberek · 2024-07-24T08:30:52Z

For aarch64-darwin:

       > [508/511] Linking C static library lib/builtins/libclang_rt.builtins_arm64_osx.a
       > [509/511] Generating ../darwin/libclang_rt.osx.a
       > FAILED: lib/darwin/libclang_rt.osx.a /tmp/nix-build-compiler-rt-libc-aarch64-apple-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/lib/darwin/libclang_rt.osx.a
       > cd /tmp/nix-build-compiler-rt-libc-aarch64-apple-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/lib/builtins && /nix/store/b4lfys8gkiwwdad8nxsdfhzvy5gzi1jw-cmake-3.29.6/bin/cmake -E make_directory /tmp/nix-build-compiler-rt-libc-aarch64-apple-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/lib/darwin && /nix/store/5pn1viwwkw60f9gcrcbcsg9792rrlpgz-llvm-binutils-16.0.6/bin/aarch64-apple-darwin-lipo -output /tmp/nix-build-compiler-rt-libc-aarch64-apple-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/lib/darwin/libclang_rt.osx.a -create -arch arm64 /tmp/nix-build-compiler-rt-libc-aarch64-apple-darwin-16.0.6.drv-0/compiler-rt-src-16.0.6/compiler-rt/build/lib/builtins/libclang_rt.builtins_arm64_osx.a
       > /bin/sh: /nix/store/5pn1viwwkw60f9gcrcbcsg9792rrlpgz-llvm-binutils-16.0.6/bin/aarch64-apple-darwin-lipo: No such file or directory
       > [510/511] Linking C static library lib/builtins/libclang_rt.cc_kext_arm64_osx.a
       > ninja: build stopped: subcommand failed.
       For full logs, run 'nix log /nix/store/9aczpyanlpj3d291d9zfxfajfvg0rs58-compiler-rt-libc-aarch64-apple-darwin-16.0.6.drv'.

RossComputerGuy · 2024-07-24T14:50:10Z

@tomberek I've pushed a change which I believe might fix it, if it doesn't then I'll do some more diagnosing tonight.

tomberek · 2024-07-25T05:49:09Z

Builds on x86_64-linux.

trofi · 2024-07-28T15:46:20Z

Bisect says 6a163e2 rustc: fix building with llvm (#320432) broke the eval of pkgsLLVM.betterbird-unwrapped on staging as:

$ nix-instantiate -A pkgsLLVM.betterbird-unwrapped
error:
       … while calling a functor (an attribute set with a '__functor' attribute)
         at lib/customisation.nix:264:13:
          263|     in if missingArgs == {}
          264|        then makeOverridable f allArgs
             |             ^
          265|        # This needs to be an abort so it can't be caught with `builtins.tryEval`,

       … while evaluating a branch condition
         at lib/customisation.nix:148:7:
          147|     in
          148|       if isAttrs result then
             |       ^
          149|         result // {

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'override' missing
       at pkgs/applications/networking/browsers/firefox/common.nix:167:18:
          166|   # Force the use of lld and other llvm tools for LTO
          167|   llvmPackages = llvmPackages0.override {
             |                  ^
          168|     bootBintoolsNoLibc = null;

RossComputerGuy · 2024-07-28T15:48:26Z

@trofi Will take a look at it tonight. Tbf, it probably wouldn't even have built anyway so it's not like we totally broke everything.

trofi · 2024-07-28T15:52:36Z

My concern is the eval only (I usually scan for things in treewide evals). It used to eval successfully and does not anymore.

RossComputerGuy · 2024-07-28T18:09:42Z

Heh yeah, I'll take a look tonight after work to try fixing this. Unless someone else wants to touch this.

The PR which fixed `useLLVM` native rustc builds (NixOS#320432) also broke cross builds. The C compiler it used for `llvmShared` and `llvmPackages`, `llvmPackages.clangUseLLVM`, was built to run on the host and create binaries for the target (`pkgsHostTarget`). Instead, the compiler should be built to run on the build machine and produce binaries for the build machine (`pkgsBuildHost`). Replace the overridden compilers with ones from `pkgsBuildHost` Co-Authored-By: Artemis Tosini <me@artem.ist>

The PR which fixed `useLLVM` native rustc builds (NixOS#320432) also broke cross builds. The C compiler it used to produce `llvmShared` and `llvmPackages`, `llvmPackages.clangUseLLVM`, was built to run on the host and create binaries for the target (`pkgsHostTarget`). Instead, the compiler should be built to run on the build machine and produce binaries for the host machine (`pkgsBuildHost`). Replace the overridden compilers with ones from `pkgsBuildHost` Co-Authored-By: Artemis Tosini <me@artem.ist>

github-actions bot added the 6.topic: rust label Jun 17, 2024

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch from e05dbb6 to 8f443be Compare June 17, 2024 05:33

RossComputerGuy marked this pull request as ready for review June 17, 2024 05:56

RossComputerGuy requested review from Mic92, zowoq, winterqt and figsoda as code owners June 17, 2024 05:56

ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild 10.rebuild-linux: 0 This PR does not cause any packages to rebuild labels Jun 17, 2024

RossComputerGuy mentioned this pull request Jun 20, 2024

Rust does not build with llvm-unwind rust-lang/rust#126752

Open

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch from 8f443be to b966a22 Compare June 22, 2024 14:16

RossComputerGuy marked this pull request as draft June 22, 2024 20:45

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch 7 times, most recently from e73636c to 1108be3 Compare June 23, 2024 02:15

ofborg bot requested a review from tjni June 23, 2024 02:16

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Jun 23, 2024

RossComputerGuy marked this pull request as ready for review July 5, 2024 21:43

RossComputerGuy marked this pull request as draft July 6, 2024 00:23

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch from 8394848 to 8b96d40 Compare July 6, 2024 00:26

RossComputerGuy changed the base branch from master to staging July 6, 2024 00:26

RossComputerGuy marked this pull request as ready for review July 6, 2024 00:26

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch from 8b96d40 to 0c3cbd2 Compare July 6, 2024 00:27

Mic92 reviewed Jul 6, 2024

View reviewed changes

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch from 0c3cbd2 to f9d3bb5 Compare July 6, 2024 06:32

ofborg bot requested a review from Mic92 July 6, 2024 09:03

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch from f9d3bb5 to 36219b4 Compare July 6, 2024 14:36

rustc: fix building with llvm

2c03b10

RossComputerGuy force-pushed the fix/pkgsllvm/rustc branch from 36219b4 to 2c03b10 Compare July 24, 2024 14:45

tomberek merged commit 6a163e2 into NixOS:staging Jul 25, 2024
24 of 26 checks passed

RossComputerGuy deleted the fix/pkgsllvm/rustc branch July 25, 2024 05:49

RossComputerGuy mentioned this pull request Jul 26, 2024

wrapRustcWith: fix linking with libgcc_s with llvm #330037

Open

13 tasks

rhelmot mentioned this pull request Sep 5, 2024

rustc: Fix cross build for FreeBSD #339909

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rustc: fix building with llvm #320432

rustc: fix building with llvm #320432

RossComputerGuy commented Jun 17, 2024 •

edited

Loading

tomberek commented Jun 20, 2024

RossComputerGuy commented Jun 20, 2024

RossComputerGuy commented Jun 22, 2024

jopejoe1 commented Jun 22, 2024

RossComputerGuy commented Jun 22, 2024

paparodeo commented Jun 22, 2024

RossComputerGuy commented Jun 23, 2024

RossComputerGuy commented Jul 5, 2024 •

edited

Loading

RossComputerGuy commented Jul 5, 2024

Mic92 Jul 6, 2024

RossComputerGuy Jul 6, 2024

Mic92 Jul 6, 2024

RossComputerGuy Jul 6, 2024

nixos-discourse commented Jul 18, 2024

tomberek commented Jul 24, 2024

RossComputerGuy commented Jul 24, 2024

tomberek commented Jul 25, 2024

trofi commented Jul 28, 2024

RossComputerGuy commented Jul 28, 2024

trofi commented Jul 28, 2024

RossComputerGuy commented Jul 28, 2024

	++ optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && useLLVM) "--push-state --as-needed -L${llvmPackages.libcxx}/lib -lc++ -lc++abi -lLLVM-${lib.versions.major llvmPackages.llvm.version} --pop-state"
	++ optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && useLLVM)
	"--push-state --as-needed -L${llvmPackages.libcxx}/lib -lc++ -lc++abi -lLLVM-${lib.versions.major llvmPackages.llvm.version} --pop-state"

	optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && !useLLVM) "--push-state --as-needed -lstdc++ --pop-state"
	optional (stdenv.isLinux && !withBundledLLVM && !stdenv.targetPlatform.isFreeBSD && !useLLVM)
	"--push-state --as-needed -lstdc++ --pop-state"

rustc: fix building with llvm #320432

rustc: fix building with llvm #320432

Conversation

RossComputerGuy commented Jun 17, 2024 • edited Loading

Description of changes

Things done

tomberek commented Jun 20, 2024

RossComputerGuy commented Jun 20, 2024

RossComputerGuy commented Jun 22, 2024

jopejoe1 commented Jun 22, 2024

RossComputerGuy commented Jun 22, 2024

paparodeo commented Jun 22, 2024

RossComputerGuy commented Jun 23, 2024

RossComputerGuy commented Jul 5, 2024 • edited Loading

RossComputerGuy commented Jul 5, 2024

Mic92 Jul 6, 2024

Choose a reason for hiding this comment

RossComputerGuy Jul 6, 2024

Choose a reason for hiding this comment

Mic92 Jul 6, 2024

Choose a reason for hiding this comment

RossComputerGuy Jul 6, 2024

Choose a reason for hiding this comment

nixos-discourse commented Jul 18, 2024

tomberek commented Jul 24, 2024

RossComputerGuy commented Jul 24, 2024

tomberek commented Jul 25, 2024

trofi commented Jul 28, 2024

RossComputerGuy commented Jul 28, 2024

trofi commented Jul 28, 2024

RossComputerGuy commented Jul 28, 2024

RossComputerGuy commented Jun 17, 2024 •

edited

Loading

RossComputerGuy commented Jul 5, 2024 •

edited

Loading